How to expose your Datastage job as a web service

It is now possible to expose the Datastage jobs you have created, as a web service. This is more in line with the SOA architecture that they are aiming for. However you simply cannot expose any job as a web service. Your jobs will have to be designed to interpret or send web service messages. The jobs can either be something that is always running or a job that runs to completion a single time. If your job is something that is meant to be triggered by web service requests then you should set that job to the multiple instance mode and select the enabled for service in the job properties window your job( as shown below). Continue Reading…

Tagged , , ,

Buffering in Datastage

Buffering is a technique used in the Datastage jobs to ensure a constant and uninterrupted flow of data to and from stages in such a way that there is no potential dead lock or any fork join problems. It is been implemented in Datastage keeping in mind the fact that the data has to keep moving in the process with an optimized use of the memory in the server. As mentioned by IBM the ideal scenario is when the data flows through the stages without being written on the disk. As in the case of buffering in any system, the upstream operators should to wait for the downstream operators to consume their input before starting to create their records. This is the intention in Datastage too.

Continue Reading…

Tagged , , , , , ,

Using the Datastage Message handlers

Datastage has a property in which it allows users to suppress or demote a warning from the Datastage log file. Although this is one thing ive never seen be used that much it’s still something worth knowing. Datastage allows users to carry out this task by the use of Message handlers. This can be done either at a job level or at a project level. Lets have a look on how to set the message handlers in Datastage.

Continue Reading…

Tagged , ,

Configuring the XML Input stage

Configuring the XML Input stage XMLs are widely used in enterprises for exchanging messages and at some point or the other you are going to have to extract the data out of XML messages. For this purpose we use the XML Input stage that is part of the real time processing stages in the palette section. The best way to explain how to use this stage is by an example. Let’s have a look at the XML I created for this purpose.

Continue Reading…

Tagged ,

Running UNIX commands in Datastage

There have always been requirements in which you would need to run certain unix commands or shell scripts within Datastage. Although not a popular demand, its still something that can lie in the ‘good to know’ category. There are a number of different ways you can actually do  this. I will try and explain the methods I have tried out.

Continue Reading…

Tagged , , , , , , ,