complex flat file stage datastage example programs

Note, however, that multi-instance jobs share slots for these files, because although each job run instance creates a separate file handle, this just increments a usage counter in the table if the file is already open to another instance. Use this command to assist with tuning the T30FILE parameter.Ī Technote has already been issued on this topic ( Every running DataStage job requires at least 3 slots in this table. The following engine command, executed from $DSHOME, shows the number of dynamic files in use: If this value is too low, expect to find an error message similar to ‘T30FILE table full’. This parameter determines the maximum number of dynamic hash files that can be opened system-wide on the DataStage system. So the dsrpcd process (23978) has four files open. Lrwx- 1 root dstage 64 Sep 25 08:24 3 -> socket: On most UNIX systems, the proc file system can be used to monitor the file handles opened by a given process for example: This will allow the DataStage process to open up to 512 – (MFILES + 8 ) files. Ideally, it would be recommended that nofiles be at least 512. NOTE: The OS parameter of nofiles needs to be set higher than MFILES. If the value is set too low, then performance issues may occur, as the server engine will make more calls to open and close at the physical OS level in order to map the logical pool to the physical pool.

Generally, a value of around 250 is suitable. Increase this value if DataStage jobs use a lot of files. The server engine will logically open and close files at the DataStage application level and physically close them at the OS level when the need arises. It does not include files opened directly at OS level by the parallel engine (PXEngine running osh). This is a per process pool for files such as sequential files that are opened by the DataStage server runtime. This parameter defines the size of the server engine (DSEngine) rotating file pool. The most commonly used parameters in the UVCONFIG file are as follows:

Complex flat file stage datastage example programs