DataStage is the ETL (Extract, Transform, Load) component of the IBM InfoSphere Information Server suite. It allows the user to integrate various data sources and targets in an enterprise environment as a GUI based client tool.
DataStage is the ETL (Extract, Transform, Load) component of the IBM InfoSphere Information Server suite. It allows the user to integrate various data sources and targets in an enterprise environment as a GUI based client tool. Data Sources/Targets could be database tables, flat files, datasets, csv files etc. Basic design paradigm consists of a unit of work called as DataStage job. Multiple jobs can be controlled and conditionally sequenced using 'Sequences'.
IBM® InfoSphere® DataStage® integrates data across multiple systems using a high performance parallel framework, and it supports extended metadata management and enterprise connectivity. The scalable platform provides more flexible integration of all types of data, including big data at rest (Hadoop-based) or in motion (stream-based), on distributed and mainframe platforms.
InfoSphere DataStage provides these features and benefits:
- Powerful, scalable ETL platform
- Support for big data and Hadoop
- Near real-time data integration
- Workload and business rules management
- Ease of use
Support for big data and Hadoop
- Includes support for IBM InfoSphere BigInsights, Cloudera, Apache and Hortonworks Hadoop Distributed File System (HDFS).
- Offers Balanced Optimization for Hadoop capabilities to push processing to the data and improve efficiency.
- Supports big-data governance including features such as impact analysis and data lineage
Powerful, scalable ETL platform
Manages data arriving in near real-time as well as data received on a periodic or scheduled basis.
Provides high-performance processing of very large data volumes.
Leverages the parallel processing capabilities of multiprocessor hardware platforms to help you manage growing data volumes and shrinking batch windows.
Supports heterogeneous data sources and targets in a single job including text files, XML, ERP systems, most databases (including partitioned databases), web services, and business intelligence tools.
Near real-time data integration
Captures messages from Message Oriented Middleware (MOM) queues using Java Message Services (JMS) or WebSphere MQ adapters, allowing you to combine data into conforming operational and historical analysis perspectives.
Provides a service-oriented architecture (SOA) for publishing data integration logic as shared services that can be reused over the enterprise.
Can simultaneously support high-speed, high reliability requirements of transactional processing and the large volume bulk data requirements of batch processing.
Ease of use
Includes an operations console and interactive debugger for parallel jobs to help you enhance productivity and accelerate problem resolution.
Helps reduce the development and maintenance cycle for data integration projects by simplifying administration and maximizing development resources.
Offers operational intelligence capabilities, smart management of metadata and metadata imports, and parallel debugging capabilities to help enhance productivity when working with partitioned data.
