Identity Manager is coming with a lot of Java classes for dealing with internal structures and objects. On top of this there are a lot of helper or utility classes, that are proven to be useful. An enhancement by self developed Java classes is often not necessary, but possible of course. In fact it is a good idea to check if a particular problem isn't already solved by use of one of the existing classes or methods.Identity Manager is coming with it's own build in programming language called XPRESS. It's meant to be used as a sort of glueing code. It's targeted for implementing smaller requirements. XPRESS is an XML based language and optimized for strings and lists, but not limited to them alone
The ingredients of an Identity Manager projects are:
Resource definitions,
Configuration objects,
Form definitions,
JSP's
Images,
Workflow definitions,
Functions and/or function libraries,
Java classes (if necessary).
Everything apart from the Java classes are XML objects and are stored in the repository. The Java classes are stored in the application filesystem and packaged into the war file. Therefore every Identity Manager project consists of two components: the WAR file and an XML import file for the repository.
Creation of deployment packages
The final application consists of the initial product as delivered to the customer and the project specific definitions, modifications and enhancements. The final package is build by merging these pieces together. First, the delivery package has to be extracted into a so called staging directory.Afterwards all project specific components (e.g. HTML pages, JSP files and Java class files) are copied to the appropriate location in the staging directory and sub hierarchy . Then the staging directory is repackaged into a WAR file (Web Application ARchive).
This file is now the program to be installed on any application server or servlet
container.
Deployment
deployment describes the installation of the WAR file into the run time container, Very often the file is installed using tools of the servlet container of your choice. Others need the file simply extracted at a special directory, monitored by the container. This feature is usually called auto deployment.
The tools just copy the WAR file to the correct location and extract the content, if needed.Depending on the servlet container a restart of the server or the container can be necessary. Now the application can be accessed using the deployment URL. This URL is constructed by using the contextroot supplied at deployment time and the containers base URL. Assuming that the contextroot was choosen to be /idm the deployment URL would be
:/idm">http://:/idm.
As already described there is a second part necessary for a complete installation of Identity Manager. The repository must be populated with the XML objects. This is a two phased job. First the default XML objects have to be imported, then the update file as build by the IDM IDE has to be incorporated. This file doesn't hold the XML objects by itself, it is just a wrapper including the different XML files. See
IDM_Installation for further guidance on how to generate the initial product WAR file. For populating the repository simple import the
/sample/init.xml file. Keep in mind, that the project specific XML objects overwrite existing objects of the same name or Id.
It's best practice to not alter the original XML objects. Instead copy the file containing the XML object of your needs to a project specific file, then change the name and id to a project specific value.
XML objects can be referenced by other XML objects using their names or ids. Therefore in some cases a simple rename might not be sufficient. If references exists, these references has to be changed accordingly.Apart from full deployments there are incremental deployments. If only modifications of XML objects were made, the only thing to do is to import the XML update file as build by the IDM IDE. If Java classes or JSP files have been modifed, the WAR file must be build and deployed again.A full deployment runs like this:
Creation of the deployment archive build from the product delivery bits, the necessary drivers, any additional Java classes and modified JSP files (if any)
Creation of the repository database tables with the creation scripts from the samples directory
Import of the product delivery XML objects
Adaption of the project specific XML objects to the target environment (change host name, login credentials, etc. according to your targeted environment)
Import of the project specific XML objects
Deployment of the created archive.
Development process
a manual installation of Identity Manager is a complex and error prone process. Historically the Business Process Editor (BPE) has been used to modify XML objects. The BPE acted directly on the objects stored in the repository. The objects were read, interactively edited and written back. As the modifications weren't tracked or stored somewhere in the filesystem it was very hard to track modifications in a consistent manner.
The only way to generate consistent, reproducible project packages was to store local copies of new or modified XML objects and use the custom build environment (CBE), a tool that was build in the field for use in projects. As the file export wasn't enforced it was usually forgotten and the content of the repository and the files on disk were not identical.The introduction of the IDM IDE with Identity Manager 6.0 addressed these aspects of the development process. It is developed as a plugin for the Netbeans development tool. As the IDE already addresses the default requirements of a developer including support for version control systems, the plugin can concentrate on aspects that are Identity Manager specific. It still relies on the CBE, but completely covers the process the developer to deal with. You can still access the repository directly with the IDM IDE and work on XML objects, but usually this should not been done.
Version control
With the introduction of IDM IDE the development process of an Identity Manager project is nearly identical to every other distributed software development project. The next issue to address is the subject of versioning and controlled access to the source files. This issue is addressed by use of a version control system. These systems track every modification with a time stamp and the authorship. Ideally every modification is documented using the check-in message, but this is often forgotten.
Deployment Environments
To ease the development, test, live cycle, it is favorable to decouple the different stages. This is done by creating so called environments. Most often a test, integration and production environment is setup.The software developer should be capable of changing whatever piece of code he wants to modify. The testers prefer to have a stable environment and stable software. And the life systems definitely shouldn't be touched without having tested and certified the software package to be released.The requirements of the environments are different and can be tweaked to the particular needs. The requirements of the development environment for example aren't that high. Very often this environment is implemented by means of Vmware Images. Depending on the selected resources, it might be necessary to choose a particular operating system or to use more than one Vmware image with different operating systems. Active Directory for example is only available on Windows Server.The test systems should mimic the production environment as closely as possible. Because you have to be able to check for network or architecture related issues. Usually this means using the same network architecture including router and load balancers, but stick with single systems for the different services. If no load tests are necessary, the use of virtual images is possible even here.The definite deployment process is different from project to project. Most customer already developed an own understanding on how a deployment should occur. Usually there are several rules. For example it could be that the developer is not allowed to test the software. Or the installation of the deployment package can only be done by the operators.A process example might run as follows: Development, code changes takes place only in the development environment. Also most of the debugging is done here. After a release candidate is developed, the package is handed over to the test team. They are deploying the release in the test environment and run the pre-documented test cases. If bugs are found, they are documented using a bug tracking tool. The process then loops back to the developers. They are in charge of fixing the bugs. If a release is considered to be stable and no more (critical) bugs show up, the release candidate is approved and is called a release. Now this release can be installed in the life system.
Object ID Management
One of the more subtle issues around producing a customized IDM release is the management of XML objects' object IDs. They are normally created automatically by IDM during the creation or import of a new XML object and therefore are not actually needed in the original XML document. There may be situations however where you need to know the object ID of one object to put it into a reference to that object inside another object.Also, you may not know from the start to what objects you may need to build references later on. Therefore it's good practice to always select an object ID manually and put it into the XML object. This ensures, that references will never go astray and makes the whole process easier to understand. Therefore it gets manageable and more consistent. In fact IDM IDE checks for the presence of Object ID's while importing and issues warnings if they are missing.The building of Object ID's should be done according to a naming policy. Stick with it throughout the project. A commonly used process would
form IDs like this: #ID#:-:.
The version is relevant here for cases, where you may need to change the object ID. You will need this when updates of TaskDefinitions for workflows with live instances are involved. IDM will not normally allow you to change those TaskDefinitions for consistency reasons.
Naming rules
If more then one person is working on a project it is common practice to define project specific naming rules. To stick with the rules helps to identify a file and it's type just by the name. If the attribute naming is consistent it is much easier to identify attributes and map them between resources and the internal namespace. It doesn't matter what the definitions are as long as every participant comply to the definitions.To be as portable as possible omit spaces and non-ascii characters in filenames, it makes life much easier.
XML files
Object Type Name Pattern
Rule Libraries
-RuleLib-.xml
Correlation Rule
-CorrelRule-.xml
Task Definitions
-Tdef-.xml
Object Ids
Object ID's (see preceeding chapter) strictly follows the following patterns, note that the version is only mandatory if it's a Task Definition. The Object Types of the above definitions is used.
#ID#:#
#ID#RuleLib:Password
#ID#CorrelRule:MatchPersonalNummer
#ID#Rule:generateAccountId
#ID#Form:ActiveSync-AD
#ID#Tdef:CreateUser#001
This environment usually consists of virtualized servers with support for all necessary resources. According to the project approach these resources are either real installations or are mimicked using flat files or simulated resources. Larger systems like, for example, a SAP system are rarely installed. A database or Active Directory is more easily incorporated. Developer don't work directly on these systems. They are used as a runtime environment.
Test Environment
This environment is used to check different aspects of the solution. At first there is the functional test of the application. Act the workflows and forms according to their specification? Secondly the initial load and the reconciliations have to be tested with data from the life system. Thirdly the data quality has to be checked. Is data cleansing necessary on the various resources before they can be connected? Next the semantics of the written data has to be checked. Are the attributes written by Identity Manager in the same format as the natively written ones? Can the local application deal with it and does it perform the way it is expected? In case of Active Directory can a newly created user account be used for logging on a windows workstation?If load tests should be conducted or the time needed for data import should be determined, this environment must mimic the production environment as close as feasible. With test cases like this it must be taken into account, that the Identity Manager performance is largely dependent on the performance of the connected resources.The test environment very often consists of genuine resources. If you are lucky as a real separate test system. If you are not so lucky as a subsystem, a cloned database table or a directory subtree on the production systems. Note that the use of extra tables or subtrees on the production systems can impact daily operations. Some Active Directory attributes for example do have impacts on the server as a whole, even if these attributes are written only to a specific subtree. The much better approach is to separate production and test environments.As the Identity Manager and especially the Access Manager are web based applications, the production environment is usually configured in a layered 3-tier architecture. This architecture has large impacts in regards to the configuration of the different systems of the over all architecture. As these configurations can be very complex, they are error prone and you would prefer to have the architecture as similar as possible. This is mainly tied to the number of different servers, SSL termination points, redirect configurations and the
configuration of the load balancers.
At the customer site the overall architecture of the production environment is usually already fixed. They have several servers around that can't be changed. A preferred vendor is selected and default server systems that have to be used are defined. Very often even the application server is defined. The project and the server components have to integrate into the existing landscape and only if there are very good technical reasons, an exception is acceptable.Very often the deployments can't be done by the project team. Then the operation admins are responsible for this. Keep in mind that they have to have the appropriate Know-How or it must be acquired on time.Another issue is to incorporate the Identity Management systems into the already existing system management and monitoring.After the deployment the initial load of the data and/or the reconciliations take place. It should be done as specified and already tested in the test environment. As this process might be quite time consuming, it is recommended to look for an appropriate time window to interfere as little as possible with daily operations. Ideally all the tests defined and tested in the test environment are applied to the freshly installed production environment.
A current PC or Laptop equipped with quite some memory, an internet connection and access to the version control system and the various environments is sufficient as a development workstation.The operating system doesn't matter that much anymore, as most of the current IDE's and tools are available for most of the popular OS. Netbeans for example is available for Solaris, Windows, different Linux flavors and Mac OS.After the initial installation of the IDE the developer should do a first check out of the project bits and pieces from the version control system. Now he is ready for creating new files, modifying or deleting existing ones. You commit the local changes and store them into the repository. So the new versions are accessible by other developers.If the developer workstation is equipped with even more memory (2 - 4+ GB), it's possible to run local target environments as virtual images on the same machine.
IDE
All files including the XML files and Java or JSP files can be created and edited with your favorite editor of choice. Stories are told that the hard core developer is still using basic vi and is quite happy with it. We might prefer to use a so called Integrated Development Environment (IDE). Tools like these come with wizards, assistants, context sensitive help, code completion and the integration of debug, profiling and other tools.Examples for IDE's are
Netbeans or
Eclipse.Identity Manager up to Identity Manager 7.1.x is coming with a bundled plugin for the Netbeans IDE. Starting with Identity Manager 8.0 it is a separate Open Source Project. The plugin can be downloaded
here. The current stable version (March 2009) is tested and supported when used with Netbeans 6.1. Nevertheless I run the same plugin succesfully with Netbeans 6.5/6.5.1. Keep in mind that it is not supported though. A version for the Eclipse 3.3.1 IDE is available at the same web site, it is however currently not a supported product.Up to Identity Manager 7.1.x the plugin is documented in
IDM_DeploymentTools. Starting with Identity Manager 8.0 the plugin should be documented with manuals located at the web site. At the time of this writing the web site documents changes from the 7.1 documentation. A full featured manual is not available yet. Therefore it is best to stick with the above document.
The use and handling of Netbeans is learned best by walking through the available tutorials and how-tos available
here.
The IDM-IDE supports to flavours of IDM projects:
Identity Manager ProjectThis is recommended as the primary project for Identity Manager deployments. It includes the CBE (Configuration Build Environment) and supports editing and debugging of Identity Manager objects, Java, and JSPs. In addition, it supports an embedded repository and is integrated with Netbean's embedded application server.
Identity Manager Project (Remote)This is a lightweight Identity Manager project. It is primarily intended for debugging and making small modifications to an external Identity Manager installation. It only supports Identity Manager objects, not Java or JSPs.
The latter project type relies on a so called IDE Compatility Bundle. This file is located as "sample/ide-bundle.zip" in your Identity Manager Installation directory. For Identity Manager 7.0, the compatility bundle may be downloaded from the Identity Manager 7.0 product download webpage.
Build ProcessAfter the creation of a new Netbeans IDM project a file "Readme.txt" is created in the project root. This file holds the most current recommendations for using the IDM IDE (partly) and the Configuration Build Environment (CBE). This file details the directory structure necessary for a Netbeans project, the build process and the different build targets.
A project directory
The directory
/idm-staging/ holds the extracted WAR file of the to be used Identity Manager version. While building the project the content is copied into the directory /idm. Afterwards the content of the subdirectory hierarchy /custom is copied to /idm. A file located in custom with the same name as a file located in idm-staging will therefore be overwritten. That means the content of /custom takes precedence over the content of /idm-staging/.
The file build.xml in the project root is the CBE/ant specific build file. It's targeted for command line oriented use with ant.The file build-config.properties holds the CBE specific configurations.
The *-target.properties files are storing the substitution patterns as described above. The patterns are applied to all files from
/custom. See the default *-target.properties files for more information.
The directory
/src is used for storing all of the project Java code. The compiled class files are located in /image/idm/WEB-INF/classes.
The directory
/custom/WEB-INF/config/ holds all XML objects. After applying the string substitution they are copied to the directory /image/idm. The import file generator is working later on on this directory hierarchy. This tool builds the file /image/idm/WEB-INF/config/custom-init-full.xml that includes every XML object file. It simplifies the import of the XML objects into the Identity Manager repository.
The files
/custom/WEB-INF/config/custom-init-preprocess.xml and /custom/WEB-INF/config/custom-init-postprocess.xml are copied to /image/idm/WEB-INF/config and can be used to deal with special situations, where XML objects have to be imported before any other object is imported or after the last object has been imported.
Additional libraries and JAR files needed by ant and/or the IDM IDE are stored in
/tools.If everything went well, the deployment files and XML import files are stored in image/. The directory /image/idm/ holds the extracted deployment file. This is the directory, that is used, when inside the IDM IDE Run Project is selected. This is also true for the Run LH Command.The /image/doc/javadoc/ directory is the container for the javadoc files, generated from the Java files located in /src.The hierarchy /image-profiler/idm/ holds a copy of /image/idm. All class and JAR files are instrumented for profiling.
ProjectName>/
/custom <-- JSP Code/custom/WEB-INF/config <-- XML Objects /custom/WEB-INF/config/AdminUser \ /custom/WEB-INF/config/Forms \ /custom/WEB-INF/config/Organization \ /custom/WEB-INF/config/Resources \ structure example /custom/WEB-INF/config/Roles / /custom/WEB-INF/config/Rules / /custom/WEB-INF/config/SystemObjects / /custom/WEB-INF/config/Workflows / /custom/WEB-INF/lib <-- jar Files /idm-staging <-- exploded idm.war /image <--deployment images /nbproject/private/src <-- Java Code
Ant, the underlying build tool, works with so called build targets. A target is a symbol, that specifies the file to be build and lists all the necessary files it depends on. If a necessary file is not available, it is automatically build as defined. A build target therefore defines a target, all necessary source files and the way how the target is build from the source.The following targets are defined in the CBE specific build.xml. There are more, but these are the most important ones:
buildThis target creates the project as a whole, but doesn't create a deployment file. The XML objects that have to be imported, are located at
/image/idm/WEB-INF/config/. The target is compiling for the default CBE target (sandbox), if it is not overwritten by a command line option or the corresponding IDM IDE property. While building, all Java files are compiled, the pattern substitution is applied to the XML files and they are checked. If configured, the JSP files are also checked. After this target has been run, a full fledged ready-to-run local Identity Manager is located in /image/idm.A subsequent Create IDM WAR (IDM IDE) or dist (ant) creates the deployment file.
Create IDM WAR / distCreates the deployment file as /image/idm-.war. If necessary a preceeding build is triggered.
cleanCleans out the project. It deletes the directories /tmp, /image and /image-profiler/.
dist-deltaA Zip file with the project specific modifications is created. It is called /image/idm-.zip.
dist-jarA Java archive with the project specific compiled Java class files is created at /image/idm-.jar. This file can be copied to /WEB-INF/lib. Then, the class files aren't necessary anymore.
jdocCreates the Javadocs from the Java files and places them into /image/doc/javadoc.
